Discontinuities, such as depth edges, in a scene play a key role in various computer vision applications such as segmentation, object detection, and pose estimation. Using a single 2D image, it is difficult to distinguish depth edges or discontinuities from intensity edges.
In U.S. Pat. No. 7,738,725, Raskar et al. describe a multi-flash camera (MFC) for extracting depth edges by casting shadows from different light positions for non-photorealistic rendering. Depth edges extracted using the MFC have been also used for silhouette-based 3D reconstruction, depth-edge-preserving stereo matching, and object detection and pose estimation in robotic and other industrial applications. All of those cameras and methods acquire multiple images by using one flash at a time for each image.
Feris et al. describe a color multiplexed MFC using (three) distinct red, green, and blue light sources for depth edge extraction. Because that method encodes shadows from the three distinct light sources separately into each RGB channel of the camera, it is only applicable to three light source positions. In addition, that method also requires a reference image acquired with white light sources. The method takes a ratio of the colored channels and the white light images.
Another method that uses a single image for depth edge extraction is based on frequency multiplexing. That method projects multiple sinusoidal patterns such that the frequencies are maintained independent of the scene geometry, and then performs frequency demultiplexing to detect shadow regions or to recover individually illuminated images. That method requires multiple projectors as light sources. That method also sacrifices spatial resolution due to frequency computation in local neighborhoods.
Illumination Multiplexing
Illumination multiplexing has been used for various active illumination applications, including photometric stereo, structured light, image-based relighting, and object material segmentation. The multiplexing is performed using conventional three red-green-blue (RGB) color channels. More channels can also be used for multispectral cameras and illuminations to reduce the number of acquired images. Multiplexing can also improve the signal-to-noise ratio using the same number of acquired images. With color and time multiplexing, at least (n+2)/3 images are required to demultiplex n light sources. The goal of those methods is to demultiplex acquired images to obtain multiple images as if the scene was illuminated by individual light sources.
Complementary Colors
Complementary colors can be used to obtain object material colors in active illumination systems. One method acquires two consecutive images by using complementary colors for each projector pixel, or each light source. The two images are added to simulate the case as if the scene was illuminated by white light sources.
Another system projects two images from two projectors such that the images have pixel-wise complementary colors on a plane. That system can be used for artistic visualization to colorize the shadow of a user interacting with the system.